Device for imaging partial fields of view, multi-aperture imaging device and method of providing same

ABSTRACT

A device includes an image sensor and an array of optical channels, each optical channel including an optic for projecting a partial field of view of a total field of view onto an image sensor area of the image sensor. A first optical channel of the array is configured to image a first partial field of view of the total field of view. A second optical channel of the array is configured to image a second partial field of view of the total field of view. The device includes a calculating unit configured to obtain image information of the first and second partial fields of view on the basis of the imaged partial fields of view, and to obtain image information of the total field of view, and to combine the image information of the partial fields of view with the image information of the total field of view so as to generate combined image information of the total field of view.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of copending InternationalApplication No. PCT/EP2018/025110, filed Apr. 11, 2018, which isincorporated herein by reference in its entirety, and additionallyclaims priority from German Application Nos. DE 10 2017 206 442.0, filedApr. 13, 2017, which is incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

The present invention relates to a device for multi-channel capturing ofa total field of view, to a supplementation device for supplementing anexisting camera, to a multi-aperture imaging device, and to methods ofproducing a device and multi-aperture imaging device described herein.The present invention further relates to a symmetric channel arrangementto different fields of view.

Conventional cameras each exhibit an imaging channel which images theentire object field. Other cameras include several imaging channels soas to image the total field of view by several partial fields of view.For correct stitching of images (sowing together and/or joining) for atotal field of view having objects which exhibit different distancesfrom the camera, a depth map of the captured total field of view mayneed to be calculated. If stereoscopic capturing is used for thispurpose, a perspective of the (artificial, central) reference camera mayneed to be synthesized. This may result in occultation or occlusionproblems since some objects may be covered up along a line of vision.For outputting a preview and/or a video, image processing, e.g. by meansof stitching, may be performed, which requires a large amount ofcalculation expenditure.

Therefore, what would be desirable is a concept of providinghigh-qualitative images which do not exhibit the above-mentioneddisadvantages.

SUMMARY

According to an embodiment, a device may have: an image sensor; an arrayof optical channels, each optical channel having an optic for projectinga partial field of view of a total field of view onto an image sensorarea of the image sensor, a first optical channel of the array beingconfigured to image a first partial field of view of the total field ofview, and a second optical channel of the array being configured toimage a second partial field of view of the total field of view; and acalculating unit configured to acquire image information of the firstand second partial fields of view on the basis of the imaged partialfields of view, and to acquire image information of the total field ofview, and to combine the image information of the partial fields of viewwith the image information of the total field of view so as to generatecombined image information of the total field of view; wherein thedevice is configured to acquire the image information of the total fieldof view having a first degree of scanning, to acquire the imageinformation of the first or second partial field of view having a seconddegree of scanning, which is larger than the first degree of scanning,and to provide the combined image information of the total field of viewhaving a third degree of scanning larger than the first degree ofscanning; or wherein the calculating unit is configured to subdivide theimage information of the total field of view and the image informationof the partial fields of view into image blocks and to associate, blockby block, image information, which is contained within a first imageblock of the total field of view, with matching image information of asecond image block of the first or second partial fields of view so asto increase, by combining the first and second image blocks, a degree ofscanning of the image information of the total field of view in thecombined image information.

According to another embodiment, a supplementation device may have aninventive device and be configured to be coupled to a camera so as toacquire therefrom the image information of the total field of view.

According to another embodiment, a multi-aperture imaging device mayhave: an image sensor; and an array of optical channels, each opticalchannel having an optic for projecting at least one partial field ofview of a total field of view onto an image sensor area of the imagesensor; a first optical channel of the array being configured to image afirst partial field of view of the total field of view, a second opticalchannel of the array being configured to image a second partial field ofview of the total field of view, and a third optical channel beingconfigured to fully image the total field of view; wherein the devicehas a calculating unit configured to acquire image information of thefirst and second partial fields of view on the basis of the imagedpartial fields of view, and to acquire image information of the totalfield of view on the basis of the imaged total field of view, and tocombine the image information of the partial fields of view with theimage information of the total field of view so as to generate combinedimage information of the total field of view; and the calculating unitis configured to subdivide the image information of the total field ofview and the image information of the partial fields of view into imageblocks and to associate, block by block, image information, which iscontained within a first image block of the total field of view, withmatching image information of a second image block of the first orsecond partial fields of view so as to increase, by combining the firstand second image blocks, a degree of scanning of the image informationof the total field of view in the combined image information; or whereinthe device is configured to acquire image information of the total fieldof view having a first degree of scanning from the sensor, to acquirethe image information of the first or second partial field of viewhaving a second degree of scanning, which is larger than the firstdegree of scanning, from the sensor, and to provide the combined imageinformation of the total field of view having a third degree of scanninglarger than the first degree of scanning.

According to another embodiment, a multi-aperture imaging device mayhave: an image sensor; and an array of optical channels, each opticalchannel having an optic for projecting at least one partial field ofview of a total field of view onto an image sensor area of the imagesensor; a first optical channel of the array being configured to image afirst partial field of view of the total field of view, a second opticalchannel of the array being configured to image a second partial field ofview of the total field of view, and a third optical channel beingconfigured to fully image the total field of view; wherein an imageformat of the total field of view corresponds to a redundancy-freecombination of the imaged first partial field of view and the imagedsecond partial field of view.

According to another embodiment, a method of providing a device may havethe steps of: providing an image sensor; arranging an array of opticalchannels, so that each optical channel has an optic for projecting atleast one partial field of view of a total field of view onto an imagesensor area of the image sensor, so that a first optical channel of thearray is configured to image a first partial field of view of the totalfield of view, and so that a second optical channel of the array isconfigured to image a second partial field of view of the total field ofview; and arranging a calculating unit such that same is configured toacquire image information of the first and second partial fields of viewon the basis of the imaged partial fields of view and to acquire imageinformation of the total field of view and to combine the imageinformation of the partial fields of view with the image information ofthe total field of view so as to generate combined image information ofthe total field of view; and such that the device is configured toacquire the image information of the total field of view having a firstdegree of scanning, to acquire the image information of the first orsecond partial field of view having a second degree of scanning, whichis larger than the first degree of scanning, and to provide the combinedimage information of the total field of view having a third degree ofscanning larger than the first degree of scanning; or such that thecalculating unit is configured to subdivide the image information of thetotal field of view and the image information of the partial fields ofview into image blocks and to associate, block by block, imageinformation, which is contained within a first image block of the totalfield of view, with matching image information of a second image blockof the first or second partial fields of view so as to increase, bycombining the first and second image blocks, a degree of scanning of theimage information of the total field of view in the combined imageinformation.

According to another embodiment, a method of providing a multi-apertureimaging device may have the steps of: providing an image sensor; andarranging an array of optical channels, so that each optical channel hasan optic for projecting at least one partial field of view of a totalfield of view onto an image sensor area of the image sensor, and so thata first optical channel of the array is configured to image a firstpartial field of view of the total field of view, so that a secondoptical channel of the array is configured to image a second partialfield of view of the total field of view, and so that a third opticalchannel is configured to fully image the total field of view; such thatthe device has a calculating unit configured to acquire imageinformation of the first and second partial fields of view on the basisof the imaged partial fields of view, and to acquire image informationof the total field of view on the basis of the imaged total field ofview, and to combine the image information of the partial fields of viewwith the image information of the total field of view so as to generatecombined image information of the total field of view; and thecalculating unit is configured to subdivide the image information of thetotal field of view and the image information of the partial fields ofview into image blocks and to associate, block by block, imageinformation, which is contained within a first image block of the totalfield of view, with matching image information of a second image blockof the first or second partial fields of view so as to increase, bycombining the first and second image blocks, a degree of scanning of theimage information of the total field of view in the combined imageinformation; or such that the device is configured to acquire imageinformation of the total field of view having a first degree of scanningfrom the sensor, to acquire the image information of the first or secondpartial field of view having a second degree of scanning, which islarger than the first degree of scanning, from the sensor, and toprovide the combined image information of the total field of view havinga third degree of scanning larger than the first degree of scanning.

According to another embodiment, a method of providing a multi-apertureimaging device may have the steps of: providing an image sensor; andarranging an array of optical channels, so that each optical channel hasan optic for projecting at least one partial field of view of a totalfield of view onto an image sensor area of the image sensor, and so thata first optical channel of the array is configured to image a firstpartial field of view of the total field of view, so that a secondoptical channel of the array is configured to image a second partialfield of view of the total field of view, and so that a third opticalchannel is configured to fully image the total field of view; such thatan image format of the total field of view corresponds to aredundancy-free combination of the imaged first partial field of viewand the imaged second partial field of view.

A finding of the present invention consists in having recognized thatthe above object can be achieved in that image information, e.g. aresolution of a total field of view, may be increased by combining samewith image information of partial fields of view of the same total fieldof view, that, however, the image information of the total field of viewis already present as, and may be used as, coarse information, and thatthe occurrence of occultation artifacts may be avoided by using theoverall image information.

In accordance with an embodiment, a device includes an image sensor andan array of optical channels. Each optical channel includes an optic forprojecting (imaging) a partial field of view of a total field of viewonto an image sensor area of the image sensor. A first optical channelof the array is configured to image a first partial field of view of thetotal field of view, and a second optical channel of the array isconfigured to image a second partial field of view of the total field ofview. The device includes a calculating unit configured to obtain imageinformation of the first and second partial fields of view on the basisof the imaged partial fields of view. The calculating unit is furtherconfigured to obtain image information of the total field of view, e.g.from a further device, and to combine the image information of thepartial fields of view with the image information of the total field ofview so as to generate combined image information of the total field ofview. By combining the image information of the partial fields of viewand of the total field of view, one obtains high-quality combined imageinformation since there is a large amount of image information. Inaddition, the image information of the total field of view enablesexerting an influence at a small amount of preprocessing expendituresince it may be displayed to a user without partial images having to bestitched.

In accordance with a further embodiment, a supplementation deviceincludes such a device and is configured to be coupled to a camera so asto obtain therefrom the image information of the total field of view.This enables supplementing already existing cameras, which may possiblybe mono-cameras, by additionally imaging the partial fields of view, sothat high-quality combined image information of the total field of viewis obtained. At the same time, the image of the camera may be utilizedfor exerting an influence on image processing since the informationregarding the total field of view already exists at least in a coarsemanner.

In accordance with a further embodiment, a multi-aperture imaging deviceincludes an image sensor, an array of optical channels, each opticalchannel including an optic for projecting at least one partial field ofview of a total field of view onto an image sensor area of the imagesensor. A first optical channel of the array is configured to image afirst partial field of view of the total field of view, a second opticalchannel of the array being configured to image a second partial field ofview of the total field of view, and a third optical channel beingconfigured to fully image the total field of view. This enables bothobtaining image information regarding the total field of view and,additionally, obtaining image information regarding the partial fieldsof view of the same total field of view, so that image areas of thepartial fields of view are scanned several times, which enables, e.g., astereoscopically generated depth map and, thus, high-quality imagegeneration. At the same time, there is the information regarding thetotal field of view in addition to the information regarding the partialfields of view, which enables the user to exert an influence without anyprevious image processing having been performed.

Further embodiments relate to a method of producing a device formulti-channel capturing of a total field of view and to a device forproviding a multi-aperture imaging device.

The embodiments mentioned enable avoiding or reducing occultation sincethe main line of vision of image of the total field of view and of thecombined image information of the total field of view is unchanged andis supplemented by the images of the partial fields of view.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be detailed subsequentlyreferring to the appended drawings, in which:

FIG. 1 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with an embodiment;

FIG. 2a-c show schematic representations of arrangements of partialfields of view within a total field of view, in accordance with anembodiment;

FIG. 3 shows a schematic perspective representation of a multi-apertureimaging device, comprising a calculation unit, in accordance with anembodiment;

FIG. 4 shows a schematic representation of image sensor areas as may bearranged, e.g., within the multi-aperture imaging device of FIG. 1 orFIG. 3, in accordance with an embodiment;

FIG. 5 shows a schematic representation of a possible implementation ofthe calculating unit, in accordance with an embodiment;

FIG. 6 shows a schematic top view of the multi-aperture imaging deviceof FIG. 3 and in accordance with an embodiment, said multi-apertureimaging device being configured to generate a depth map;

FIG. 7 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with a further embodiment which includes displaymeans;

FIG. 8 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with an embodiment which comprises an opticalstabilizer and an electronic image stabilizer;

FIG. 9 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with a further embodiment which includes focusingmeans;

FIG. 10 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with a further embodiment, wherein the image sensorareas are arranged on at least two mutually different chips and areorientated against one another;

FIG. 11 shows a schematic perspective view of a multi-aperture imagingdevice in accordance with a further embodiment, wherein optics havedifferent optical lengths;

FIG. 12 shows a schematic perspective view of a device in accordancewith a further embodiment;

FIG. 13 shows a schematic perspective view of a supplementation devicein accordance with an embodiment;

FIG. 14 shows a schematic flow chart of a method of providing a devicein accordance with an embodiment; and

FIG. 15 shows a schematic flow chart of a method of providing amulti-aperture imaging device in accordance with an embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Before embodiments of the present invention will be explained in moredetail with reference to the drawings, it shall be noted that elements,objects and/or structures which are identical, identical in function oridentical in action have been provided with identical reference numeralsin the different figures, so that the descriptions of said elementswhich have been presented in the different embodiments areinterchangeable and/or are mutually applicable.

FIG. 1 shows a schematic perspective view of a multi-aperture imagingdevice 10 in accordance with an embodiment. The multi-aperture imagingdevice 10 includes an image sensor 12 comprising a plurality of imagesensor areas 24 a-c. The image sensor 12 may be implemented such thatthe image sensor areas 24 a-c are part of a shared chip, butalternatively, it may also comprise several components, which means thatthe image sensor areas 24 a-c may be arranged on different chips.Alternatively or additionally, the image sensor areas 24 a and 24 clocated opposite the image sensor area 24 b, and/or the image sensorarea 24 a located opposite the image sensor area 24 b may have differentsizes in terms of the sensor surface and/or different numbers and/orsizes of pixels.

The multi-aperture imaging device 10 further includes an array 14 ofoptical channels 16 a-c. Each of the optical channels 16 a-c includes anoptic 64 a-c for projecting at least a partial field of view of a totalfield of view or of an object area onto an image sensor area 24 a-c ofthe image sensor 12. One of the optics 64 a-c is associated with one ofthe image sensor areas 24 a-c, respectively, and is configured toinfluence an optical path 26 a-c, e.g. by means of concentration orscattering, so that the respective partial field of view or total fieldof view is projected onto the image sensor area 24 a-c. The optics 64a-c may be arranged on a shared carrier so as to form the array 14, butthey may also be mechanically connected to one another in a differentmanner or may not be in mechanical contact. Properties of the opticalchannels such as length, extension perpendicular to the optical axis orthe like, and/or properties of the optics, e.g. focal length, f-number,aperture diameter, aberration correction or physical dimension may varyamong the optical channels 16 a, 16 b and/or 16 c and may differ fromone another.

Two of the optical channels 16 a-c are configured to project one partialfield of view onto the associated image sensor area 24 a-c,respectively. Projecting (imaging) of a partial field of view here meansthat the total field of view is imaged in an incomplete manner. Afurther optical channel of the optical channels 16 a-c is configured tofully image the total field of view. The multi-aperture imaging device10 is implemented, for example, such that the optical channel 16 b isconfigured to fully capture the total field of view. The opticalchannels 16 a and 16 c are configured, for example, to capture suchpartial fields of view of the total field of view which overlap eachother in an incomplete manner at the most or which are arranged in amutually disjoint manner. This means that the arrangement of the optics64 a and 64 c for capturing the first and second partial fields of viewin the array 14 may be symmetric in relation to the optic 64 b forcapturing the imaging of the total field of view and/or that thearrangement of the image sensor areas 24 a and 24 c for imaging thefirst and second partial fields of view may be symmetric in relation toa location of the image sensor area 24 b for imaging the total field ofview. Even though any other associations between fields of view, opticsand image sensor areas are also possible, the symmetric arrangement, inparticular, offers the advantage that additional capturing of thepartial fields of view enables symmetric disparity with regard to thecentral field of vision, i.e. of capturing the total field of view.

The multi-aperture imaging device 10 may comprise optionalbeam-deflecting means 18 including, in turn, beam-deflecting areas 46a-c, the beam-deflecting means 18 being configured to deflect, with eachof the beam-deflecting areas 46 a-c, an optical path 26 a-c. Thebeam-deflecting means 18 may include a mirror surface comprising thebeam-deflecting areas 46 a-c. Alternatively, at least two of thebeam-deflecting areas 46 a-c may be mutually tilted and form a pluralityof mirror surfaces.

Alternatively or additionally, the beam-deflecting means 18 may comprisea plurality or multitude of facets. Utilization of the beam-deflectingmeans 18 may be advantageous if the field of view to be captured islocated in a direction of the multi-aperture imaging device 10 whichdiffers from the line of vision between the image sensor 12 and thearray 14. Alternatively, if the beam-deflecting means 18 is not present,the total field of view may be captured along the line of vision of themulti-aperture imaging device 10 and/or the direction between the imagesensor 12 and the array 14 and beyond. Arranging the beam-deflectingmeans 18, however, may enable changing of the line of vision of themulti-aperture imaging device 10 by moving the beam-deflecting means 18in a translational and/or rotational manner, without having to changethe spatial orientation of the image sensor 12 and/or of the array 14for this purpose.

FIG. 2a shows a schematic representation of an arrangement of partialfields of view 72 a and 72 b within a total field of view 70, which maybe captured, e.g., by the multi-aperture imaging device 10. For example,the total field of view 70 may be projected, by using the opticalchannel 16 b, onto the image sensor area 24 b. For example, the opticalchannel 16 a may be configured to capture the partial field of view 72 aand to project it onto the image sensor area 24 a. The optical channel16 c may be configured to capture the partial field of view 72 b and toproject it onto the image sensor area 24 c. This means that a group ofoptical channels may be configured to capture precisely two partialfields of view 72 a and 72 b.

Even though the partial fields of view 72 a and 72 b are depicted tohave different extensions in order to improve distinguishability, theymay have identical or comparable extensions along at least an imagedirection 28 or 32, e.g. along the image direction 32. The extension ofthe partial fields of view 72 a and 72 b may be identical with theextension of the total field of view 70 along the image direction 32.This means that the partial fields of view 72 a and 72 b may fullycapture the total field of view 70 along the image direction 32 and maycapture the total field of view in an only partial manner along adifferent image direction 28 arranged perpendicularly to the former, andmay be arranged to be offset from one another, so that completecapturing of the total field of view 70 also results along the seconddirection by means of combination. Here, the partial fields of view 72 aand 72 b may be disjoint in relation to one another or may overlap in anincomplete manner, at the most, in an overlap area 73, which possiblyfully extends along the image direction 32 in the total field of view70. A group of optical channels including the optical channels 16 a and16 c may be configured to fully image the total field of view 70 whencombined. The image direction 28 may be a horizontal, e.g., of an imageto be provided. In simplified terms, the image directions 28 and 32represent two different image directions which are spatially arranged inany manner desired.

FIG. 2b shows a schematic representation of an arrangement of thepartial fields of view 72 a and 72 b, which are arranged to be mutuallyoffset along a different image direction, namely the image direction 32,and which mutually overlap. The partial fields of view 72 a and 72 b mayfully capture the total field of view 70 along the image direction 28,and in an incomplete manner along the image direction 32, respectively.The overlap area 73 is fully arranged, e.g., within the total field ofview 70 along the image direction 28.

FIG. 2c shows a schematic representation of four partial fields of view72 a to 72 d, which capture the total field of view 70 in an incompletemanner in both directions 28 and 32, respectively. Two adjacent partialfields of view 72 a and 72 b overlap in an overlap area 73 b. Twooverlapping partial fields of view 72 b and 72 c overlap in an overlaparea 73 c. Similarly, partial fields of view 72 c and 72 d overlap in anoverlap area 73 d, and the partial field of view 72 d overlaps with thepartial field of view 72 a in an overlap area 73 a. All four partialfields of view 72 a to 72 d may overlap in an overlap area 73 e of thetotal field of view 70.

For capturing the total field of view 70 and the partial fields of view72 a-d, a multi-aperture imaging device may be configured similarly towhat was described in the context of FIG. 1, wherein the array 14 maycomprise, e.g., five optics, four for capturing partial fields of view72 a-d, and one optic for capturing the total field of view 70.

A large number of image information items are available in the overlapareas 73 a to 73 e. For example, the overlap area 73 b is captured viathe total field of view 70, the partial field of view 72 a and thepartial field of view 72 b. An image format of the total field of viewmay correspond to a redundancy-free combination of the imaged partialfields of view, e.g., partial fields of view 72 a-d in FIG. 2c , inwhich context the overlap areas 73 a-e are counted only once in eachcase. In connection with FIGS. 2a and 2b , this applies toredundancy-free combination of the partial fields of view 72 a and 72 b.An overlap in overlap areas 73 and/or 73 a-e may include, for example,50% at the most, 35% at the most, or 20% at the most of the respectivepartial images.

FIG. 3 shows a schematic perspective representation of a multi-apertureimaging device 30 in accordance with a further embodiment, which expandsthe multi-aperture imaging device 10 by a calculating unit 33.

The calculating unit 33 is configured to obtain image information fromthe image sensor 12, which means image information regarding the partialfields of view, e.g., partial fields of view 72 a and 72 b, which havebeen projected onto the image sensor areas 24 a and 24 c, as well asimage information of the total field of view, e.g., the total field ofview 70, which may be projected onto the image sensor area 24 b. Thecalculating unit 33 is configured to combine the image information ofthe partial fields of view and the image information of the total fieldof view. Combining of the image information may be performed, e.g., suchthat the degree of scanning of the total field of view is lower than adegree of scanning of the partial fields of view. A degree of scanningmay be understood to mean a local resolution of the partial or totalfield of view, i.e., a quantity which indicates which surface in theobject area is projected onto which surface area or pixel size of theimage sensor. In implementations described herein, the term resolutionis understood to mean the extension of the partial or total field ofview that is projected onto a corresponding image sensor surface. Acomparatively larger resolution thus means that a constant surface areaof a field of view, given an identical pixel size, is projected onto alarger image sensor surface, and/or that a comparatively smaller objectsurface area is projected, given an identical pixel size, onto aconstant image sensor surface. By combining the image information, adegree of scanning and/or a resolution of the combined image information61 may be increased in relation to capturing of the total field of view.

FIG. 4 shows a schematic representation of image sensor areas 24 a-c asmay be arranged, e.g., within the multi-aperture imaging device 10 or30. The partial field of view 72 a is projected onto the image sensorarea 24 a, for example. The partial field of view 72 b is projected ontothe image sensor area 24 c, for example. The total field of view 70 isprojected onto the image sensor area 24 b, for example. The spatialarrangement of the partial fields of view 72 a and 72 b may correspondto the configuration of FIG. 2b , for example.

The image sensor areas 24 a, 24 b and 24 c may have, along the imagedirection 32, a physical extension b which is identical or is identicalwithin a tolerance range of 20%, 10% or 5%, and which may correspond toa same number of pixels. Along the image direction 28, the image sensorareas 24 a and 24 c may have a physical extension a, which maycorrespond to an identical number of a pixels. The extension or pixels amay be larger, along the image direction 28, than the extension ornumber of pixels c of the image sensor area 24 b. Since the partialfields of view 72 a and 72 b are equal in size along the image direction28 and compared to the total field of view 70, scanning takes place at ahigher resolution or at a higher degree of scanning of the total fieldof view along the image direction 28, i.e., a smaller area in the objectarea is projected onto a pixel of a constant size, so that the resultingcombinational resolution, and/or the degree of scanning, is increased. Asuper-resolution effect may be implemented, for example, when the pixelsof the images of the partial fields of view exhibit a mutual subpixeloffset.

Along the image direction 32, e.g., a number of 2×b pixels are used soas to image the total field of view 70 via the partial fields of view 72a and 72 b, the overlap area 73 having to be taken into account here.The disjoint or only partial overlap of the partial fields of view 72 aand 72 b, however, also results, along the image direction 32, in aresolution which is increased as compared to capturing of the totalfield of view 70 in the image sensor area 24.

Thus, the combined image information of the total field of view 61 maybe increased, as compared to the resolution obtained in the imaging area24 b, by combining the images in the image sensor areas 24 a to 24 c. Anaspect ratio of the image in the image sensor area 24 b may have a valueof 3:4. This enables obtaining the combinational image with an identicalaspect ratio. A resolution in the image sensor areas 24 a and/or 24 cmay be larger, along the respective image direction and/or in thenascent image, than in the image sensor area 24 b, within a tolerancerange of 20%, 10%, or exactly by at least 30%, at least 50%, or at least100%; here, an extension of the overlap area is to be taken intoaccount.

The image sensor areas 24 a-c may be arranged along a line extensiondirection 35, which may be arranged, e.g., in parallel with the imagedirection 28 and/or along which the optics 64 a-c of the multi-apertureimaging device 10 or 30 may be arranged. Along a direction z, which isperpendicular thereto and which may be a thickness direction of themulti-aperture imaging device, for example, the image sensor areas 24a-c may have an extension which is identical within the tolerance range,which means that the increase in the resolution of the capturing of thetotal field of view may be obtained while avoiding an additionalthickness of the multi-aperture imaging device.

In other words, a linear symmetric arrangement of at least three camerachannels, i.e., optical channels, may be implemented, wherein one of theoptical channels, advantageously the central optical channel, covers thetotal field of view, and the (two) outer channels each cover only partof the field of view, e.g., the top/bottom or the left/right, so thattogether they also cover the total field of view and at the same timemay have a slight overlap in the center of the field of view. This meansthat on the left/right and/or at the top/bottom, high-resolution partialimages are obtained. In the center, a lower-resolution image, whichcovers the entire relevant field of view, is captured. The resolution inthe central image may be reduced to an extent as dictated or enabled bythe correspondingly shorter focal length for the same image height, i.e.without any consideration of the aspect ratio, and by the same pixelsize. In other words, the image sensor heights 24 a, 24 b and 24 c areidentical. Without any overlap, therefore, the image height in the imagesensor area 24 b is only half a combined image height of 24 a and 24 c.To image the same field of view, the focal length (or magnification) ofthe optic for the image sensor area 24 b (optical channel 26 b) maytherefore be half the length or size as for 24 a and 24 c. Given thesame pixel size, this means half the resolution (or scanning of thefield of view) in 24 b as compared to 24 a and 24 c combined.Corresponding image widths simply result from the desired aspect ratiosof the images.

The central camera channel is the reference camera for generating thedepth map if the latter is created by the calculating unit. Saidarrangement, which includes the symmetry with regard to the centralchannel, enables a high quality of the obtained combined total imagewith regard to occultation in the depth map. Therefore, it makes sensethat the central image is also the reference for calculating thehigher-resolution combined image. The at least two higher-resolutionimages are inserted block by block into the low-resolution reference.They therefore serve as material which may be employed when the accuracyof fit is ensured, i.e., when matching features are found in the partialfields of view and the total field of view. Said insertion may beeffected in very small blocks, so that even with fine objects comprisinglarge depth jumps, parallax-related problems may be avoided. Thesuitable blocks are searched for by means of correspondence, forexample, which may mean that a disparity map, i.e., a depth map, isgenerated. However, if, for a low-resolution block, no high-resolutionblock is found with sufficient reliabilty, this will have nocatastrophic effects.

One simply leaves the low-resolution original image as it is. In otherwords: holes present in the depth map will merely result in more blurredplaces in the total image rather than in clearly visible artifacts.

In other words, the central image sensor may capture, due to a shorterfocal length, a lower-resolution image which covers the total field ofview and inherently exhibits the desired aspect ratio. Said camera mayalso be referred to as a reference camera, for example since theobtained combined image exhibits said camera's perspective. What followsis a joining of higher-resolution partial images, which may partiallyoverlap and will, in combination, exhibit a same aspect ratio as thereference camera.

Joining of the images of the partial fields of view by using thereference camera enables highly correct stitching in the overlap areasince what is present there is an image which is inherently correct,even though it has a lower resolution. The heights of all three imagesensors are, in accordance with an advantageous implementation,identical or almost identical so as to exploit available design heightsin an optimum manner. All cameras may be deflected via a shared mirror(beam-deflecting means). A depth map may be calculated as follows asneed be. In the overlap area 73 of the two high-resolution partialimages, said calculation may be performed by the latter and thelow-resolution total image; in the remaining areas, said calculation maybe effected by combining one of the high-resolution partial images withthe corresponding portion of the low-resolution total image.

FIG. 5 shows a schematic representation of a possible implementation ofthe calculating unit 33. The calculating unit 33 may be configured tosplit up the image information of the total field of view 70 and theimage information of the partial fields of view 72 a and 72 b into imageblocks 63 a of the partial field of view 72 a, 63 b and of the partialfield of view 72 b and 63 c of the total field of view 70. An imageblock may comprise a specific number of pixels along both imagedirections 28 and 32. The blocks may have, along the image directions 28and 32, e.g., a size of a minimum of 2 and a maximum of 1000 pixels, ofa minimum of 10 and a maximum of 500 pixels, or of a minimum of 20 and amaximum of 100 pixels.

The calculating unit may be configured to associate, block by block,image information, which is contained within an image block of the totalfield of view, with matching image information of an image block of thefirst or second partial field of view 72 a or 72 b so as to increase aresolution of the image information of the total field of view in thecombined image information by combining the first and second imageblocks. The first and second image blocks may each be a matching imageblock of an image of different partial fields of view in an overlap areathereof. Alternatively or additionally, the first or second block may bea block of the total image, and the other block may be a block of thepartial image. The calculating unit 33 is configured, for example, toidentify the object, depicted as x in block 63 a ₃, as matching theobject x in block 63 c ₁ of the total field of view 70. On the basis ofthe resolution of the partial field of view 72 a, which is higher ascompared to that of the total field of view 70, the calculating unit maycombine the image information of both blocks 63 a ₃ and 63 c ₁ with eachother so as to obtain, in the block, a resulting resolution that ishigher than it originally was with regard to the total field of view. Inthis context, the resulting combinational resolution may correspond toor be higher than the value of the resolution of the capturing of thepartial field of view 72 a. The object depicted by # in a block 63 c ₂of the total field of view 70 is identified by the calculating unit,e.g., in a block 63 a ₂ of the partial field of view 72 a and in a block63 b _(1 of the partial field of view 72) b, so that in order to improvethe image quality, image information from both images of the partialfields of view 72 a and 72 b may be used.

An object depicted by * in a block 63 c ₃ is identified, by thecalculating unit, for example in a block 63 b ₂ of the partial field ofview 72 b, so that the image information of the block 63 b ₂ isemployed, e.g., by the calculating unit in order to increase the imageinformation in block 63 c ₃.

In a case where no block of the partial fields of view 72 a and 72 b canbe associated with a block of the total field of view, as depicted,e.g., for the block 63 c ₄, the calculating unit may be configured tooutput a block of the combined total image such that at least the block63 c ₄ is located within the total image. I.e., image information may bedepicted even if there is no local increase in the resolution. This willresult, at the most, in minor changes in the total image, which meansthat at the location of the block 63 c ₄, there will be a locallyreduced resolution, for example.

The calculating unit 33 may be configured to perform stitching of theimage information of the partial fields of view 72 a and 72 b on thebasis of the image information of the total field of view 70. This meansthat total imaging of the total field of view may be employed for atleast supporting or even performing mutual alignment of the partialimages of the partial fields of view 72 a and 72 b. Alternatively oradditionally, the information from the total imaging of the total fieldof view may be exploited for supporting or even performing arrangementof the objects of the scene from the partial images and/or within apartial image within the total image. The total field of view comprisesa large number of or even all of the objects which are also depicted inthe partial fields of view 72 a and 72 b, so that a comparison of therespective partial images with the total image of the total field ofview and/or a comparison of the position of the object will enablealignment in relation to the total image and will therefore enablemutual stitching of the partial images.

In other words, the low-resolution image will a priori provide a basisfor supporting stitching of the high-resolution images, i.e., a basisfor orientation, since objects are already present in a joined manner inthe total image. In addition to simple joining of two global partialimage areas, stitching may also mean that objects will be re-integratedinto the scene and/or against the background differently (with regard totheir lateral positions within the image) as a function of theirdistance within a stitched image, which may be required or desired,depending on the distance-related object distribution within the scene.The concepts described here simplify the stitching process considerably,even if a depth map should be required for exact stitching. Occultationproblems caused by there being no camera in the central position may beavoided since at least three optical channels enable at least threelines of vision toward the total field of view. Occultation in one angleof view may thus be reduced or prevented by one or two other angles ofview.

FIG. 6 shows a schematic top view of the multi-aperture imaging device30 in accordance with an embodiment. The calculating unit 33 may beconfigured to generate a depth map 81. The depth map 81 may refer to theimage information of the total field of view 70. The calculating unit 33is configured, for example, to exploit disparities 83 a between theimages of the partial field of view 72 a and of the total field of view70 and 73 b and between the images of the partial field of view 72 b andthe total field of view 70 in order to generate a depth map. This meansthat due to the physical distance of the optics 64 a, 64 b and 64 c aswell as of the image sensor areas 24 a, 24 b and 24 c, different anglesof view, or perspectives, are obtained which will be utilized forgenerating the depth map 81 on the part of the calculating unit 33. Thecalculating unit 33 may be configured to generate the depth map 81 inthe overlap area 73, within which the partial fields of view 72 a and 72b overlap, while using the image information of the partial fields ofview 72 a and 72 b. This enables utilization of a disparity which islarger as compared to the individual disparities 83 a and 83 b, insimplified terms, the sum of individual disparities, as well asutilization of the high-resolution (partial) images. This means, in oneembodiment, the calculating unit 33 may be configured to generate thedepth map 81 in the overlap area 73 without the information of the totalfield of view 70 projected onto the image sensor area 24 b.Alternatively, utilization of the information of the total field of view70 in the overlap area 73 is possible and is advantageous, for example,in order to achieve a high information density.

The projections in the image sensor areas 24 a and 24 c may be effected,in accordance with an advantageous further development, withoututilizing (e.g. RGB) Bayer color filter arrangements as in 24 b or atleast while using uniform color filters, so that the imaged first andsecond partial fields of view having single-color luminance informationwill be provided by the multi-aperture imaging device. For example, asingle-color infrared filter, ultraviolet filter, red filter, bluefilter or the like or no filter at all may be arranged, while there isno a multi-color filter such as a Bayer arrangement as in 24 b. In otherwords, since the outer channels only contribute details in terms ofincreasing the quality of the image of the total field of view, it maybe advantageous for the outer channels 16 a and 16 c to comprise nocolor filters. The outer channels will then contribute only luminanceinformation, i.e., an increase in the general sharpness/in details, butno improved color information; the advantage obtained therewith lies inthe increased sensitivity and, thus, in reduced noise, which for itspart eventually also enables improved resolution and/or sharpness sincethe image will have to be smoothened less since, e.g., there is no Bayercolor filter pattern superimposed on the pixels; however, the resolutionpresent in the luminance-only channels is inherently higher (ideallydouble) since no de-Bayering is necessary. Effectively, a color pixelmay be approximately double the size of a black-and-white pixel. Thismay not apply in physical terms since here, black-and-white pixelscannot be used for resolution only, but may also be used for colordiscrimination by superimposing black-and-white pixels with the typicalRGBG filter pattern.

FIG. 7 shows a schematic perspective view a multi-aperture imagingdevice 71 in accordance with a further embodiment, which includesdisplay means 85. The multi-aperture imaging device 71 is configured toreproduce the representation of the total field of view 70, which isprojected onto the image sensor area 24 b, with the display means 85. Tothis end, the calculating unit 33 may be configured, for example, toforward the corresponding signal from the image sensor 12 to the displaymeans 85. Alternatively, the display means 85 may also be directlycoupled to the image sensor 12 and obtain the corresponding signal fromthe image sensor 12.

The display means 85 is configured to receive and output the imageinformation of the total field of view at that resolution, at the most,which is provided by the image sensor area 24 b. Advantageously, theresolution of the total field of view projected onto the image sensorarea 24 b will be forwarded unchanged to the display means 85. Thisenables displaying the image or video which has possibly been currentlycaptured, for example as a preview for a user, so that said user mayinfluence capturing. The higher-resolution images provided by thecombined image information 61 may be provided to the display means 85 ata different point in time, may be provided to a different display means,may be stored or transmitted. It is also possible to obtain the combinedimage information 61 from time to time, i.e., only if and when required,and to otherwise utilize that image of the total field of view 70 whichpossibly has a lower resolution, if this is sufficient for currentutilization, e.g., viewing on the display device 85, without a depth mapbeing required or when zooming into details is dispensed with. Thisenables an influence to be exerted on image capturing without anycombination of the image signals, which would require a certain amountof expenditure in terms of calculation and time, which will have anadvantageous effect on the delay in the display means 85 and on theenergy required for the calculations. By stringing together severalimages on the part of the multi-aperture imaging device 71, it is alsopossible to obtain a video signal from the image sensor 12 and to outputa video signal of the total field of view on the display means 85.

For example, the multi-aperture imaging device 71 or a differentmulti-aperture imaging device described herein, for example themulti-aperture imaging device 10, 30 or 60, may be formed as a mobilephone, a smartphone, a tablet or a monitor.

The multi-aperture imaging device 71 may provide a real-time preview onthe display 85; the two outer camera channels therefore need not alwaysbe activated, so that current may be saved and/or that no additionalcomputing expenditure is required for linking the partial images, whichenables reduced capacitor utilization of the processor and reducedenergy consumption, which also enables extended battery life.

Alternatively or additionally, raw data may at first be stored, and ahigh-resolution image may be generated not until transmission isperformed to a different calculating unit such as a PC and/or when saidimage is viewed on the display device while zooming into the details.Here it is possible to generate the combinational image only forrelevant image areas, or to not generate the combinational image, atleast in areas, for irrelevant image areas. Relevant areas may be, e.g.,image areas for which magnified depiction (zoom) is desired.

Both for individual images (frames) and for a video, the image of theimage sensor area 24 b may thus be directly used and possibly has aresolution sufficient for a video. It is also conceivable for thecentral camera to be provided with an resolution suitable for commonvideo formats, i.e., for example, 1080p or 4K, from the start, so thatresampling (sampling-rate conversion), binning (combining of adjacentimage elements) or skipping (skipping of pixels), which is otherwisecommonly performed, may be avoided; in this context, the resolution maybe high enough so that high-resolution still pictures may be generated.

FIG. 8 shows a schematic perspective view of a multi-aperture imagingdevice 80 comprising an optical image stabilizer 22 and an electronicimage stabilizer 41. The aspects which will be described below in termsof image stabilization may be realized, without any limitations,individually or in combination while using the functionalities of thecalculating unit 33.

The optical image stabilizer 22 includes, e.g., actuators 36 a, 36 b and42, the actuators 36 a and 36 b being configured to achieve opticalimage stabilization of the images of the partial fields of view in theimage sensor areas 24 a to 24 c by displacing the array 14 along theline extension direction 35. In addition, the optical image stabilizer22 is configured, for example, to obtain optical image stabilizationalong the image axis 32 by means of a rotational movement 38 of thebeam-deflecting means 18. For example, the optics 64 a and 64 b of thearray 14 comprise effective focal lengths f₁ and f₃, respectively, whichdiffer from each other within a tolerance range of a maximum of 10%, amaximum of 5% or a maximum of 3%, so as to capture the partial fields ofview in a more or less identical manner. The optic 64 b may have a focallength f₂, which differs therefrom by at least 10%. The rotationalmovement 38 performed for all channels results, in cooperation with thedifference in focal lengths f₂ and f₁, or within the differences infocal length between f₁ and f₃, in different displacements 69 ₁ to 69 ₃of the images in the image sensor areas 24 a-c. This means that theoptical image stabilizer 22 achieves, by means of the rotationalmovement 38 performed for all channels, different effects in the images,so that at least one, several or all of the images deviate from atheoretical defect-free state. The optical image stabilizer 22 may beconfigured to globally minimize the deviations of all of the images,which may result in that defects arise in each of the images, however.Alternatively, the optical image stabilizer 22 may be configured toselect a reference image in one of the image sensor areas 24 a-d and tocontrol the actuator 42 such that the image in the reference image orreference channel is as exact as possible, which may also be referred toas defect-free. This means that by means of optical image stabilizationperformed for all channels, a channel may be kept defect-free inrelation to the image direction influenced, while the other channelsdeviate from said reference image due to the different focal lengths f₁to f₃. In other words, a channel is corrected by means of the mechanicaloptical image stabilizer that is implemented, which will have an effecton all of the channels but will not keep all of the channels stable.Said further channels will be additionally corrected by means of theelectronic image stabilizer.

The optical image stabilizer may be configured to provide the relativemovements for the optical channels in a channel-specific manner and/orindividually for groups of optical channels, e.g., for the group of theoptical channels 16 a and 16 c for capturing the partial fields of view,and for the group including the optical channel 16 b for capturing thetotal field of view.

The electronic image stabilizer 41 may be configured to performchannel-specific electronic image stabilization in each channel inaccordance with a specified functional correlation which depends on therelative movements between the image sensor 12, the array 14 and thebeam-deflecting means 18. The electronic image stabilizer 41 may beconfigured to stabilize each image individually. To this end, theelectronic image stabilizer 41 may use global values, e.g., the cameramovement or the like, so as to increase the optical quality of theimages. It is particularly advantageous for the electronic imagestabilizer 41 to be configured to perform electronic image correction onthe basis of a reference image of the optical image stabilizer 22. Thedifferent focal lengths may provide the functional correlation betweenthe different changes in the images caused by the optical imagestabilization in a advantageously linear form, e.g., in the form:

aberration=f(f _(i),relative movement),

i.e., the aberration, globally or in relation to the reference channel,may be depicted as a function of the focal length or the differences infocal length and of the relative movement performed in order to changethe line of vision or to achieve optical image stabilization. Theelectronic image stabilizer 41 may link an extent of the relativemovement between the image sensor 12, the array 14 and thebeam-deflecting means 18 to the focal lengths f₁ to f₃ or to thedifferences in focal lengths in relation to the reference channel inorder to obtain reliable information on the electronic imagestabilization to be performed, and in order to establish and/or exploitthe functional correlation. The data of the optical properties and/or ofthe functional correlation may be obtained during calibration. Mutualalignment of images for determining displacement of an image in relationto another image may also be effected by determining a matching featurein the images of the partial fields of view, e.g., edge contours, objectsizes or the like. This may be identified, e.g., by the electronic imagestabilizer 41, which may further be configured to provide electronicimage stabilization on the basis of a comparison of movements of thefeatures in the first and second images. Channel-specific electronicimage stabilization may thus be effected by means of channel-specificimage evaluation of the movements of image details.

Alternatively or additionally to a comparison in different images, it isalso possible to perform a comparison of the feature within the sameimage, in particular regarding two pictures or frames that are takenwith a time interval between been. The optical image stabilizer 41 maybe configured to identify a matching feature in the correspondingpartial image at a first point in time and at a second point in time,and to provide electronic image stabilization on the basis of acomparison of movements of the feature in the first image. Thecomparison may indicate, e.g., a stretch of displacement by which thefeature has been displaced by means of a relative movement and by whichthe image will have to be displaced back in order to at least partlycorrect the image artifact.

The optical image stabilizer may be used for stabilizing an image of theimaged partial field of view of a reference channel, e.g., the image inthe image sensor area 24 a. This means that the reference channel may befully optically stabilized. In accordance with embodiments, a pluralityof optical image stabilizers may be arranged which provide optical imagestabilization for at least groups of optical channels, e.g., the opticalchannel or optical channels having a first focal length, such as theoptical channel for imaging the total field of view, and opticalchannels having a second focal length, e.g., for imaging the partialfields of view. Alternatively, channel-specific optical imagestabilization may also be provided. The electronic image stabilizer 41is configured, e.g., to perform image stabilization in achannel-specific manner for optical channels which differ from thereference channel and which project onto the image sensor areas 24 b and24 c. The multi-aperture imaging device may be configured to stabilizethe reference channel in an exclusively optical manner. I.e., in oneembodiment, sufficient image stabilization may be achieved in thereference channel by using solely the optical image stabilization whichis achieved mechanically. For the other channels there will beadditional electronic image stabilization so as to fully or partlycompensate for the above-described effect of insufficient optical imagestabilization due to differences in the focal length, said electronicstabilization being performed individually in each channel.

In accordance with a further embodiment it is also possible for eachchannel of the multi-aperture imaging device to comprise individualelectronic image stabilization. The electronic image stabilization whichis performed individually, i.e., to a dedicated extent, for each channelof the multi-aperture imaging device may be effected such that aspecified functional correlation between the image displacements to beimplemented in the individual channels is exploited. For example, thedisplacement along the direction 32 within a channel amounts to 1.1times, 1.007 times, 1.3 times or 2 or 5 times the displacement along thedirection 32 in a different image. Furthermore, this channel-specificfunctional correlation may depend on the relative movements between thebeam-deflecting unit and/or the array and/or the image sensor, whichfunctional correlation may be linear or correspond to an angularfunction which images an angle of rotation of the beam-deflecting meansonto an extent of the electronic image stabilization along the imagedirection. An identical correlation may be obtained with identical ordifferent numerical values for the direction 28.

What applies to all embodiments is that the implemented relativemovements may be captured by corresponding additional sensors such asgyroscopes and others, for example, or may be derived from the capturedimage data of one, several or all of the channels. Said data orinformation may be used for the optical and/or electronic imagestabilizer, i.e., the multi-aperture imaging device is configured, forexample, to receive a sensor signal from a sensor and to evaluate thesensor signal with regard to information correlating with a relativemovement between the multi-aperture imaging device and the object, andto control the optical and/or electronic image stabilizer while usingsaid information.

The optical image stabilizer may be configured to obtain optical imagestabilization along the image axes 28 and 32 by moving differentcomponents, e.g., the array 14 for stabilization along the direction 28,and rotation 38 of the beam-deflecting means 18 for stabilization alongthe direction 32. In both cases, differences in the optics 64 a-c havean effect. The previous explanations regarding electronic imagestabilization may be implemented for both relative movements. Inparticular, considering the directions 28 and 32 separately from eachother enables taking into account different deviations between theoptics 64 a-c along the directions 28 and 32.

Embodiments described herein may share an image axis 28 and/or 32 forthe partial images in the image sensor areas 24 a-c. Alternatively, thedirections may also differ and be converted to each other.

FIG. 9 shows a schematic perspective view of a multi-aperture imagingdevice 90 in accordance with a further embodiment, which includesfocusing means 87. The focusing means 87 may include one or moreactuators 89 a, 89 b and/or 89 c, which are configured to change adistance between the array 14 and the image sensor 12 and/or between thebeam-deflecting means 18 and the array 14 and/or between thebeam-deflecting means 18 and the image sensor 12, so as to adjustfocusing of the images onto the image sensor areas 24 a, 24 b and/or 24c. Even though the optics 64 a, 64 b and 64 c are depicted to bearranged on a shared carrier so as to be movable together, at least theoptic 64 b, the image sensor area 24 b and/or the beam-deflecting area46 b may be moved individually so as to adjust focusing for the opticalchannel 16 b such that it differs from focusing in other channels. I.e.,the focusing means 87 may be configured to adjust a relative movementfor the first and second optical channels 16 a and 16 c and a relativemovement for the optical channel 16 b such that said relative movementsdiffer from one another.

The focusing means 87 may be combined with the optical image stabilizer22, i.e., a movement provided both in the optical image stabilizer 22and in the focusing means 87 by actuators may be provided byadditionally arranged actuators or also by a shared actuator whichprovides movements between components both for the purpose of focusingand for optical image stabilization.

In other words, utilization of separate actuators for autofocus (AF) andpossibly optical image stabilization (OIS) is advantageous. Because ofthe possibly different architectures of the adjacent channels withregard to resolution and focal length, channel-specific actuation mayenable obtaining channel-specific adjustment, so that the advantages ofautofocusing and/or of image stabilization in all of the channels may beobtained. For example, different image-side distances covered arerequired for focusing purposes for the function of autofocus atdifferent focal lengths so as to perform same with high quality.Alternative structural forms may be implemented such that the opticalchannel which is configured to capture the total field of view isconfigured without any beam-deflecting means.

FIG. 10 shows a schematic perspective representation of a multi-apertureimaging device 100 in accordance with a further embodiment, wherein theimage sensor areas 24 a to 24 c are arranged on at least two chips whichdiffer from one another and are oriented, i.e., tilted against oneanother. The image sensor area 24 b may comprise, in combination withthe optic 64 b, a first line of vision possibly directly to the totalfield of view 70. The image sensor areas 24 a and 24 c may comprise, incombination with the optics 64 a and 64 c associated with them, a lineof vision which differs from the former, e.g., is perpendicular theretoalong an x direction, the optical paths 26 a and 26 c being deflectedtoward the partial fields of view 72 a and 72 b by the beam-deflectingmeans 18. This represents a structural form which is alternative to thepreviously described multi-aperture imaging devices.

Utilization of the beam-deflecting means 18 may result in a certainmirror size or deflection surface area size, which may be larger forchannel 24 b than for the adjacent channels for capturing the partialfields of view since the channel 16 b after all has to capture the totalfield of view, which is larger than the partial fields of view 72 a and72 b. This may result in an increase in size along a thickness directionof the device, e.g. along a z direction, which in some embodiments isundesired. Utilization of the beam-deflecting means 18 may therefore bereconfigured such that only the optical paths 26 a and 26 c aredeflected, while the optical path 26 b is directed directly, i.e.without any deflection, toward the total field of view 70.

In other words, the central camera channel is centrally mounted, withoutany deflection mirrors, i.e., in a classical orientation, between thetwo deflected higher-resolution camera channels such that it directlylooks out from the plane of the device, e.g., a telephone. Due to therelatively low resolution, e.g., of a value of 0.77 and/or 1/1.3, 0.66and/or 1/1.5, or 0.5 and/or ½, which corresponds to an above-describedhigher resolution of the additional channels of at least 30%, at least50% or at least 100%, and due to a correspondingly shorter focal length,the central camera channel comprises, in such a stand-up configuration,an approximately same design height along the z direction as the twoouter camera channels have in a lying configuration. Said solution maypossibly prevent switching of the line of vision of the central channel16 b, which may be compensated for, however, by possibly also arrangingan additional camera channel. Arranging an autofocus function and/oroptical image stabilization may be provided by arranging actuatorsindividually. In yet other words: a large field of view “1” may beimaged “standing up” with a short focal length and/or small amount ofmagnification, and a smaller partial field of view “2” may be imaged “ina lying position and with a folded optical path” with a longer focallength and/or a larger amount of magnification and may be best adaptedto the respective circumstances. “1” is already designed to be short,but enables a large field of view, which, however, may also render themirror large, while “2” may be designed to be long and, due to thesmaller field of view, requires a smaller mirror.

FIG. 11 shows a schematic perspective view of a multi-aperture imagingdevice 110 in accordance with a further embodiment, wherein a distanced₁ of the optics 64 a and 64 c, which comprise the focal lengths f₁ andf₃, from the image sensor 12 is larger than a distance d₂ between theoptic 64 b and the image sensor 12, the optic 64 b comprising the focallength f₂. The distance d₁ and/or d₂ may thus be adapted to the focallengths of the optics 64 a to 64 c. Beam-deflecting areas 46 a to 46 cof the beam-deflecting means 18 may be individually controllable if thebeam-deflecting means 18 is arranged.

FIG. 12 shows a schematic perspective view of a device 120 in accordancewith a further embodiment. The device 120 includes the image sensor 12including the image sensor areas 24 a and 24 c. The device 120 furtherincludes the array 14 comprising the optical channels 16 a and 16 c.Each of the optical channels 16 a and 16 c unchangedly comprises theoptic 64 a and 64 c, respectively, for imaging a partial field of view72 a and 72 b, respectively, of the total field of view 70, as wasdescribed in connection with above-described multi-aperture imagingdevices. In simplified terms, the image sensor 12 and the array 14 maybe configured in absence of the optical channel for imaging the totalfield of view. The device 120 includes the calculating unit 33, which isconfigured to obtain image information regarding the partial fields ofview 72 a and 72 b from the image sensor 12. The calculating unit 33 isfurther configured to obtain a signal 91 including image information ofthe total field of view 70. The calculating unit 33 is configured tocombine the image information of the partial fields of view 72 a and 72b with the image information 91 of the total field of view 70 so as toobtain the combined image information 61 of the total field of view 70.

In simplified terms, the device 120 may be an additional module for anexisting camera device and may be configured to obtain, from the cameradevice, the image signal regarding the captured total field of view.Said camera device may be any camera. The device 120 may thus beconfigured to increase a resolution of the external device in that theobtained image signal 91 is superposed with additionally capturedpartial fields of view 72 a and 72 b in order to improve the quality.

The calculating unit 33 may be configured to provide the samefunctionality as is described in connection with the multi-apertureimaging devices described herein. This means that the device 120 may beconfigured to obtain the image information 91 at a first imageresolution, and to obtain the image information regarding the partialfields of view 72 a and 72 b at a higher resolution. The combined imageinformation 61 may comprise the higher resolution or at least a higherresolution than that of the image signal 91. Along first and secondimage directions, the resolution of the image signal 91 may correspondto or be smaller than, within the above-described tolerance range of20%, 10% or 0%, the resolution of the image sensor areas 24 a and 24 cmultiplied by the values 0.77, 0.66, and/or ½.

The calculating unit 33 may further be configured to perform said imageassociation block by block, as is described in connection with FIG. 5.This may also mean that the calculating unit 33 performs an associationcriterion, e.g., a similarity analysis, an edge comparison or the like,and performs combining of the blocks only if the association criterionis met. If the association criterion is not met, the combined imageinformation may be provided by the calculating unit 33 such thatcombining of the blocks in the evaluated block is not performed.

The calculating unit 33 may be configured to perform stitching of theimage information of the partial fields of view 72 a and 72 b on thebasis of the image information of the total field of view 70. Since onlythe image information is evaluated, it may be irrelevant to thecalculating unit 33 whether or not the image information regarding thetotal field of view 70 is obtained from the system's own image sensor 12or from the image sensor of an external device.

As was described for the multi-aperture imaging devices, the calculatingunit may be configured to generate a depth map for the image informationof the total field of view while using a first disparity between a lineof vision of the channel for capturing the total field of view and animage of the first partial field of view and/or a second disparitybetween the image of the total field of view and an image of the secondpartial field of view. Even though the total field of view was capturedby a different camera, it comprises a line of vision toward the totalfield of view, so that a disparity can be evaluated, in particular ifthe device is calibrated together with the device providing the imagesignal 91.

As was described above, the calculating unit 33 may be configured togenerate the depth map in an overlap area of the partial images of thepartial fields of view 72 a and 72 b by exclusively using the imageinformation of the partial fields of view.

The device 120 may comprise an optical image stabilizer, e.g., theoptical image stabilizer 22, which is configured to provide opticalimage stabilization by generating a relative movement between the imagesensor 12 and the array 14 along first and second directions, e.g.,along image directions 28 and 32. Alternatively or additionally, thedevice 120 may include focusing means, e.g., focusing means 87, so as toadjust a focus of the device 120. This may be effected by generating arelative movement between at least one of the optics 64 a and 64 b ofthe optical channels 16 a and 16 b and the image sensor 12.

FIG. 13 shows a schematic perspective view of a supplementation device130 in accordance with an embodiment. The supplementation device 130 isconfigured to supplement a camera or image capturing device 93, thecamera or image capturing device 93 being configured to generate theimage signal 91 and to provide it to the supplementation device 130. Forexample, the supplementation device 130 includes the device 120 and isconfigured to be coupled to the camera 93. Thus, the supplementationdevice 130 is configured to expand or to supplement image processing ofthe camera 93. This may also be understood to be an add-on device which,e.g., provides a pair of outer channels as resolution boosts to anexisting system. This means that the above-described central channel maybe a conventional camera module which the supplier, e.g., of a mobilephone, knows and understands and for which there exists an establishedsupply chain. The outer channels comprising the optics 64 a and 64 c areadditional modules which are installed next to or around the former. Afolded optical path may remain, the design height thus adjusts to theold camera module. Even though a design height of the overall systemwill not be thinner than the camera module 33, an increase in the designheight may also be avoided by increasing the resolution by means of theadditional module. Instead, the outer channels together have a largersensor area than the inner module. If the pixels are equal in size, themodule will thus have a higher overall resolution and, consequently, afiner angular resolution. Therefore, it may be used for increasing theresolution as compared to the camera 93. Here, too, the sensors may beconfigured, in embodiments, without any color filters. This means thatthe image information of the imaged first and second partial fields ofview may be provided having single-color luminance information. Theadditional modules may, but need not necessarily, be assembledsymmetrically around the camera module of the camera 93. Other forms ofassembly such as diagonal or asymmetric, for example, are alsoconceivable and may be compensated for via image processing. Depth mapsmay be generated unchangedly, optical and/or electronic imagestabilization as well as autofocus may be provided.

FIG. 14 shows a schematic flowchart of a method 1400 for providing adevice in accordance with an embodiment, e.g., the device 120. A step1410 comprises providing an image sensor. A step 1420 comprisesarranging an array of optical channels, so that each optical channelincludes an optic for projecting at least one partial field of view of atotal field of view onto an image sensor area of the image sensor, sothat a first optical channel of the array is configured to image a firstpartial field of view of the total field of view, and so that a secondoptical channel of the array is configured to image a second partialfield of view of the total field of view. A step 1430 comprisesarranging a calculating unit such that same is configured to obtainimage information of the first and second partial fields of view on thebasis of the imaged partial fields of view and to obtain imageinformation of the total field of view and to combine the imageinformation of the partial fields of view with the image information ofthe total field of view so as to generate combined image information ofthe total field of view.

FIG. 15 shows a schematic flowchart of a method 1500 for providing amulti-aperture imaging device in accordance with an embodiment, e.g.,multi-aperture imaging device 10. A step 1510 comprises providing animage sensor. A step 1520 comprises arranging an array of opticalchannels such that each optical channel includes an optic for projectingat least one partial field of view of a total field of view onto animage sensor area of the image sensor, and so that a first opticalchannel of the array is configured to image a first partial field ofview of the total field of view, so that a second optical channel of thearray is configured to image a second partial field of view of the totalfield of view, and so that a third optical channel is configured tofully image the total field of view.

The embodiments described herein provide the advantage that alow-resolution image is still used a priori for supporting stitching ofthe high-resolution images so as to simplify the stitching process. Thisallows less pronounced instances of occultation in the depth map becauseof the central arrangement of the camera and the symmetric arrangementof the other channels around same. This also results in fewer artifactsin the final combined image. A live view may be directly obtained,without any computing expenditure, from the central channel, possiblywhile using binning or skipping of pixels, so as to reduce theresolution to a required extent, but the complete field of view in thecorrect aspect ratio may already be obtained from the central channelsince said central channel covers the entire FOV. A video may bedirectly derived from the central channel without any computingexpenditure, which is effected by analogy with deriving images.Utilization of only three channels, i.e., a group of optical channelsfor capturing the partial fields of view, includes optical channels forcapturing precisely two partial fields of view, allows a smaller numberof components, fewer sensors, a small data transmission bandwidth and asmall volume of the device or multi-aperture imaging device.

Embodiments described herein may be used as or in multi-aperture imagingsystems with linear channel arrangement and smallest design heightswhile offering the advantages described herein as compared to knownsolutions.

Some explanations relate to relative directions such as top/bottom orleft/right. It is understood that same are interchangeable as desired ifspatial orientation is changed. This is why said terms should not betaken to be limiting and are intended to improve clarity only.

In the following, additional embodiments and aspects of the inventionwill be described which can be used individually or in combination withany of the features and functionalities and details described herein.

According to an embodiment, a device comprises: an image sensor 12; anarray 14 of optical channels 16 a-b, each optical channel 16 a-bincluding an optic 64 a-b for projecting a partial field of view 72 a-bof a total field of view 70 onto an image sensor area 24 a-b of theimage sensor 12, a first optical channel 16 a of the array 14 beingconfigured to image a first partial field of view 72 a of the totalfield of view 70, and a second optical channel 16 b of the array 14being configured to image a second partial field of view 72 b of thetotal field of view 70; and a calculating unit 33 configured to obtainimage information of the first and second partial fields of view on thebasis of the imaged partial fields of view 72 a-b, and to obtain imageinformation of the total field of view 70, and to combine the imageinformation of the partial fields of view with the image information ofthe total field of view so as to generate combined image information 61of the total field of view.

According to a second embodiment when referring back to the firstembodiment, the first and second optical channels 16 a-b are part of agroup of optical channels configured to image one partial field of view72 a-b of the total field of view 70, respectively, the group of opticalchannels being configured to jointly fully image the total field of view70.

According to a third embodiment when referring back to the secondembodiment, the group of optical channels is configured to captureprecisely two partial fields of view 72 a-b.

According to a fourth embodiment when referring back to any one of thepreceding embodiments, a first partial image and a second partial image,which represent the image information of the first and second partialfields of view, comprise, along a first image direction 32, a samedimension as a total image representing the image information of thetotal field of view 70, and comprise, along a second image direction 28,a different dimension as compared to the total image.

According to a fifth embodiment when referring back to any one of thepreceding embodiments, the device is configured to provide the imageinformation of the imaged first and second partial fields of view havingsingle-color luminance information.

According to a sixth embodiment when referring back to any one of thepreceding embodiments, the device is configured to obtain the imageinformation of the total field of view 70 having a first degree ofscanning b×c, to obtain the image information of the first or secondpartial field of view 72 a-b having a second degree of scanning b×a,which is larger than the first degree of scanning b×c, and to providethe combined image information of the total field of view 70 having athird degree of scanning larger than the first degree of scanning.

According to a seventh embodiment when referring back to the sixthembodiment, the second degree of scanning b×c is at least 30% larger,along first and second image directions 28, 32, than the first degree ofscanning.

According to an eighth embodiment when referring back to any one of thepreceding embodiments, the calculating unit 33 is configured tosubdivide the image information of the total field of view 70 and theimage information of the partial fields of view into image blocks 63 a,63 b and to associate, block by block, image information x, *, #, whichis contained within a first image block 63 c of the total field of view,with matching image information of a second image block 63 a, 63 b ofthe first or second partial fields of view so as to increase, bycombining the first and second image blocks, a degree of scanning of theimage information of the total field of view 70 in the combined imageinformation.

According to a ninth embodiment when referring back to the eighthembodiment, the calculating unit 33 is configured to associate the firstblock 63 c with the second block 63 a, 63 b while applying anassociation criterion so as to perform the combination only if theassociation criterion is met, and to provide the combined imageinformation 61 of the total field of view 70 within the first block 63 c₄ without any combination if the association criterion is not met.

According to a tenth embodiment when referring back to any one of thepreceding embodiments, the calculating unit 33 is configured to performstitching of the image information of the partial fields of view 72 a-bon the basis of the image information of the total field of view 70.

According to an eleventh embodiment when referring back to any one ofthe preceding embodiments, the calculating unit 33 is configured togenerate a depth map for the image information of the total field ofview 70 while using a first disparity 83 a between an image of the totalfield of view 70 and an image of the first partial field of view 72 aand a second disparity 83 b between the image of the total field of view70 and an image of the second partial field of view 72 b.

According to an twelfth embodiment when referring back to any one of thepreceding embodiments, the partial fields of view 72 a-b overlap withinan overlap area 73; 73 a-e within the total field of view 70, thecalculating unit 33 being configured to generate a depth map for theimage information of the total field of view 70 within the overlap area73; 73 a-e while using the image information of the first and secondpartial fields of view 72 a-b.

According to a thirteenth embodiment when referring back to any one ofthe preceding embodiments, the device comprises an optical imagestabilizer 22 for image stabilization along a first image axis 28 bygenerating a first relative movement 34; 39 a between the image sensor12 and the array 14 and for image stabilization along a second imageaxis 32 by generating a second relative movement 38; 39 b between theimage sensor 12 and the array 14.

According to a fourteenth embodiment when referring back to any one ofthe preceding embodiments, the device further comprises focusing meansincluding at least one actuator 134 b for adjusting a focus of thedevice, said actuator 134 b being configured to provide a relativemovement between at least one optic 64 a-b of one of the opticalchannels 16 a-b and the image sensor 12.

According to a fifteenth embodiment, a supplementation device comprisesa device according to any one of the first to fourteenth embodiments andis configured to be coupled to a camera so as to obtain therefrom theimage information of the total field of view 70.

According to a sixteenth embodiment, a multi-aperture imaging device 10;30; 60; 70; 80; 90 comprises: an image sensor 12; and an array 14 ofoptical channels 16 a-b, each optical channel 16 a-b including an optic64 a-c for projecting at least one partial field of view 72 a-b of atotal field of view 70 onto an image sensor area 24 a-c of the imagesensor 12; a first optical channel 16 a of the array 14 being configuredto image a first partial field of view 72 a of the total field of view70, a second optical channel 16 b of the array 14 being configured toimage a second partial field of view 72 b of the total field of view 70,and a third optical channel 16 c being configured to fully image thetotal field of view 70.

According to a seventeenth embodiment when referring back to thesixteenth embodiment, the multi-aperture imaging device includes abeam-deflecting means 18 for jointly deflecting an optical path 26 a-bof the first and second optical channels 16 a-b.

According to an eighteenth embodiment when referring back to thesixteenth or seventeenth embodiment, an arrangement of optics 64 a, 64 cfor capturing the first and second partial fields of view in the array14 is symmetric in relation to a location of the optic 64 b for imagingthe total field of view 70; or wherein an arrangement of image sensorareas 24 a, 24 c for imaging the first and second partial fields of view72 a-b is symmetric in relation to a location of the image sensor area24 b for imaging the total field of view 70.

According to a nineteenth embodiment when referring back to any one ofthe sixteenth to eighteenth embodiments, an image format of the totalfield of view 70 corresponds to a redundancy-free combination of theimaged first partial field of view 72 a and the imaged second partialfield of view 72 b.

According to a twentieth embodiment when referring back to any one ofthe sixteenth to nineteenth embodiments, the multi-aperture imagingdevice includes a calculating unit 33 configured to obtain imageinformation of the first and second partial fields of view on the basisof the imaged partial fields of view 72 a-b, and to obtain imageinformation of the total field of view 70 on the basis of the imagedtotal field of view 70, and to combine the image information of thepartial fields of view with the image information of the total field ofview 70 so as to generate combined image information 61 of the totalfield of view 70.

According to a twenty-first embodiment when referring back to thetwentieth embodiment, the calculating unit 33 is configured to subdividethe image information of the total field of view 70 and the imageinformation of the partial fields of view into image blocks and toassociate, block by block, image information, which is contained withina first image block of the total field of view 70, with matching imageinformation of a second image block of the first or second partialfields of view so as to increase, by combining the first and secondimage blocks, a degree of scanning of the image information of the totalfield of view 70 in the combined image information.

According to a twenty-second embodiment when referring back to thetwenty-first embodiment, the calculating unit 33 is configured toassociate the first block with the second block while applying anassociation criterion so as to perform the combination only if theassociation criterion is met, and to provide the combined imageinformation of the total field of view 70 within the first block withoutany combination if the association criterion is not met.

According to a twenty-third embodiment when referring back to any one ofthe twentieth to twenty-second embodiments, the calculating unit 33 isconfigured to perform stitching of the image information of the partialfields of view on the basis of the image information of the total fieldof view 70.

According to a twenty-fourth embodiment when referring back to any oneof the twentieth to twenty-third embodiments, the calculating unit 33 isconfigured to generate a depth map 81 for the image information of thetotal field of view 70 while using a first disparity 83 a between animage of the total field of view 70 and an image of the first partialfield of view and a second disparity 83 b between the image of the totalfield of view 70 and an image of the second partial field of view.

According to a twenty-fifth embodiment when referring back to any one ofthe twentieth to twenty-fourth embodiments, the partial fields of viewoverlap within an overlap area within the total field of view 70, thecalculating unit 33 being configured to generate a depth map for theimage information of the total field of view 70 within the overlap areawhile using the image information of the first and second partial fieldsof view.

According to a twenty-sixth embodiment when referring back to any one ofthe twentieth to twenty-fifth embodiments, the multi-aperture imagingdevice is configured to obtain image information of the total field ofview 70 having a first degree of scanning from the sensor, to obtain theimage information of the first or second partial field of view having asecond degree of scanning, which is larger than the first degree ofscanning, from the sensor, and to provide the combined image informationof the total field of view 70 having a third degree of scanning largerthan the first degree of scanning.

According to a twenty-seventh embodiment when referring back to thetwenty-sixth embodiment, the second degree of scanning is at least 30%larger, along first and second image directions, than the first degreeof scanning.

According to a twenty-eighth embodiment when referring back to any oneof the sixteenth to twenty-seventh embodiments, the multi-apertureimaging device is configured to obtain image information of the totalfield of view 70, which is captured by the third optical channel, fromthe image sensor at a first image resolution, the multi-aperture imagingdevice including display means and being configured to display the imageinformation at the first image resolution at the most.

According to a twenty-ninth embodiment when referring back to thetwenty-eighth embodiment, the multi-aperture imaging device isconfigured to output a video signal of the total field of view 70 on thedisplay means on the basis of consecutive images of the total field ofview 70.

According to a thirtieth embodiment when referring back to any one ofthe sixteenth to twenty-ninth embodiments, the multi-aperture imagingdevice includes beam-deflecting means 18 for shared deflecting of anoptical path 26 a-c of the optical channels 16 a-c, and comprising anoptical image stabilizer 22 for image stabilization along a first imageaxis 28 by generating a first relative movement between the image sensor12, the array 14 and the beam-deflecting means 18, and for imagestabilization along a second image axis 32 by generating a secondrelative movement 38 between the image sensor 12, the array 14 and thebeam-deflecting means 18.

According to a thirty-first embodiment when referring back to thethirtieth embodiment, the optical image stabilizer 22 is configured toprovide a first relative movement for the first and second opticalchannels and a second relative movement for the third optical channel.

According to a thirty-second embodiment when referring back to thethirtieth or thirty-first embodiment, the first relative movementincludes at least one of a translational relative movement between theimage sensor 12 and the array 14, a translational relative movementbetween the image sensor 12 and the beam-deflecting means 18, and atranslational relative movement between the array 14 and thebeam-deflecting means 18, and wherein the second relative movement 38includes at least one of a rotational movement of the beam-deflectingmeans 18, a translational relative movement between the image sensor 12and the array 14, and a translational relative movement between thearray 14 and the beam-deflecting means 18.

According to a thirty-third embodiment when referring back to any one ofthe thirtieth to thirty-second embodiments, the multi-aperture imagingdevice further comprises an electronic image stabilizer 41 for imagestabilization of the first optical channel 16 a of the array 14 alongthe first and second image axes 28, 32.

According to a thirty-fourth embodiment when referring back to thethirty-third embodiment, the electronic image stabilizer 41 isconfigured to stabilize the first optical channel 16 a to a first extentalong the first and second image axes 28, 32, and further is configuredfor image stabilization of the second optical channel 16 c to a secondextent along the first and second image axes 28, 32.

According to a thirty-fifth embodiment when referring back to thethirty-third or thirty-fourth embodiment, the optical image stabilizer22 is configured to perform optical image stabilization such that saidoptical image stabilization is related to an image of a first one of thepartial fields of view 72 a-b, wherein the electronic image stabilizer41 is configured to stabilize an image of a second partial field of view72 a-b in relation to the image of the first partial field of view 72a-b.

According to a thirty-sixth embodiment when referring back to any one ofthe thirty-third to thirty-fifth embodiments, the optical imagestabilizer 22 is configured to stabilize an image of the imaged partialfield of view 72 a-b of a reference channel from a group including thefirst optical channel 16 a and the second optical channel 16 c, andwherein the electronic image stabilizer 41 is configured to performimage stabilization in a channel-specific manner for optical channels 16a-c which differ from the reference channel, the multi-aperture imagingdevice being configured to stabilize the reference channel in anexclusively optical manner.

According to a thirty-seventh embodiment when referring back to any oneof the thirty-third to thirty-sixth embodiments, the electronic imagestabilizer 41 is configured to perform image stabilization for eachoptical channel 16 a-c in a channel-specific manner.

According to a thirty-eighth embodiment when referring back to thethirty-seventh embodiment, the electronic image stabilizer 41 isconfigured to perform channel-specific electronic image stabilization ineach channel in accordance with a specified functional correlation whichdepends on the relative movements between the image sensor 12, the array14 and the beam-deflecting means 18.

According to a thirty-ninth embodiment when referring back to thethirty-eighth embodiment, the functional correlation is a linearfunction.

According to a fortieth embodiment when referring back to any one of thethirty-third to thirty-ninth embodiments, the optical image stabilizer22 is configured to provide the optical image stabilization along one ofthe image directions 28, 32 on the basis of a rotational movement 38 ofthe beam-deflecting means 18, the functional correlation being anangular function which projects an angle of rotation of thebeam-deflecting means 18 onto an extent of the electronic imagestabilization along the image direction 28, 32.

According to a forty-first embodiment when referring back to any one ofthe thirty-third to fortieth embodiments, the electronic imagestabilizer 41 is configured to identify a matching feature in a firstpartial image of a first partial field of view 72 a-b at a first pointin time and at a second point in time, and to provide the electronicimage stabilization on the basis of a comparison of movements of thefeature in the first image.

According to a forty-second embodiment when referring back to any one ofthe sixteenth to forty-first embodiments, the multi-aperture imagingdevice further includes focusing means 87 including at least oneactuator 89 a-b for adjusting a focus of the device, said actuator beingconfigured to provide a relative movement between at least one optic 64a-c of one of the optical channels 16 a-c and the image sensor 12.

According to a forty-third embodiment when referring back to theforty-second embodiment, the focusing means 87 is configured to providea third relative movement for the first and second optical channels 16a, 16 c and a fourth relative movement for the third optical channel 16b.

According to a forty-fourth embodiment when referring back to any one ofthe sixteenth to forty-third embodiments, the image sensor areas arearranged on the image sensor 12 along a line extension direction 35, andwherein the image sensor areas 24 a-c exhibit, along an image direction32 perpendicular to the line extension direction, dimensions which areidentical within a tolerance range of 20%.

According to a forty-fifth embodiment when referring back to any one ofthe sixteenth to forty-fourth embodiments, the first and second opticalchannels 16 a, 16 c are part of a group of optical channels configuredto image a partial field of view 72 a-c of the total field of view 70,respectively, said group of optical channels being configured to jointlyfully image the total field of view 70.

According to a forty-sixth embodiment when referring back theforty-fourth embodiment, the group of optical channels is configured tocapture precisely two partial fields of view 72 a-b.

According to a forty-seventh embodiment when referring back to any oneof the sixteenth to forty-sixth embodiments, a first partial image and asecond partial image, which represent the image information, provided bythe image sensor 12, of the first and second partial fields of view 72a-b, have a same dimension, along a first image direction 32, as a totalimage which represents image information, provided by the image sensor,of the total field of view 70, and exhibit, along a second imagedirection 28, a different dimension as compared to the total image.

According to a forty-eighth embodiment when referring back to any one ofthe sixteenth to forty-seventh embodiments, the multi-aperture imagingdevice is configured to provide the imaged first and second partialfields of view 72 a-b having single-color luminance information.

According to a forty-ninth embodiment when referring back to any one ofthe sixteenth to forty-eighth embodiments, the multi-aperture imagingdevice is configured as a mobile phone, a smartphone, a tablet, or amonitor.

According to a fiftieth embodiment, a method 1400 of providing a devicecomprises: providing 1410 an image sensor; arranging 1420 an array ofoptical channels, so that each optical channel includes an optic forprojecting at least one partial field of view of a total field of viewonto an image sensor area of the image sensor, so that a first opticalchannel of the array is configured to image a first partial field ofview of the total field of view, and so that a second optical channel ofthe array is configured to image a second partial field of view of thetotal field of view; and arranging 1430 a calculating unit such thatsame is configured to obtain image information of the first and secondpartial fields of view on the basis of the imaged partial fields of viewand to obtain image information of the total field of view and tocombine the image information of the partial fields of view with theimage information of the total field of view so as to generate combinedimage information of the total field of view.

According to a fifty-first embodiment, a method 1500 of providing amulti-aperture imaging device comprises: providing 1510 an image sensor;and arranging 1520 an array of optical channels, so that each opticalchannel includes an optic for projecting at least one partial field ofview of a total field of view onto an image sensor area of the imagesensor, and so that a first optical channel of the array is configuredto image a first partial field of view of the total field of view, sothat a second optical channel of the array is configured to image asecond partial field of view of the total field of view, and so that athird optical channel is configured to fully image the total field ofview.

Even though some aspects have been described within the context of adevice, it is understood that said aspects also represent a descriptionof the corresponding method, so that a block or a structural componentof a device is also to be understood as a corresponding method step oras a feature of a method step. By analogy therewith, aspects that havebeen described within the context of or as a method step also representa description of a corresponding block or detail or feature of acorresponding device.

While this invention has been described in terms of several embodiments,there are alterations, permutations, and equivalents which fall withinthe scope of this invention. It should also be noted that there are manyalternative ways of implementing the methods and compositions of thepresent invention. It is therefore intended that the following appendedclaims be interpreted as including all such alterations, permutationsand equivalents as fall within the true spirit and scope of the presentinvention.

1. A device comprising: an image sensor; an array of optical channels, each optical channel comprising an optic for projecting a partial field of view of a total field of view onto an image sensor area of the image sensor, a first optical channel of the array being configured to image a first partial field of view of the total field of view, and a second optical channel of the array being configured to image a second partial field of view of the total field of view; and a calculating unit configured to acquire image information of the first and second partial fields of view on the basis of the imaged partial fields of view, and to acquire image information of the total field of view, and to combine the image information of the partial fields of view with the image information of the total field of view so as to generate combined image information of the total field of view; wherein the device is configured to acquire the image information of the total field of view comprising a first degree of scanning, to acquire the image information of the first or second partial field of view comprising a second degree of scanning, which is larger than the first degree of scanning, and to provide the combined image information of the total field of view comprising a third degree of scanning larger than the first degree of scanning; or wherein the calculating unit is configured to subdivide the image information of the total field of view and the image information of the partial fields of view into image blocks and to associate, block by block, image information, which is comprised by a first image block of the total field of view, with matching image information of a second image block of the first or second partial fields of view so as to increase, by combining the first and second image blocks, a degree of scanning of the image information of the total field of view in the combined image information.
 2. The device as claimed in claim 1, wherein the first and second optical channels are part of a group of optical channels configured to image one partial field of view of the total field of view, respectively, the group of optical channels being configured to jointly fully image the total field of view.
 3. The device as claimed in claim 2, wherein the group of optical channels is configured to capture precisely two partial fields of view.
 4. The device as claimed in claim 1, wherein a first partial image and a second partial image, which represent the image information of the first and second partial fields of view, comprise, along a first image direction, a same dimension as a total image representing the image information of the total field of view, and comprise, along a second image direction, a different dimension as compared to the total image.
 5. The device as claimed in claim 1, configured to provide the image information of the imaged first and second partial fields of view comprising single-color luminance information.
 6. The device as claimed in claim 1, configured to acquire the image information of the total field of view comprising the first degree of scanning, wherein the second degree of scanning is at least 30% larger, along first and second image directions, than the first degree of scanning.
 7. The device as claimed in claim 1, wherein the calculating unit is configured to subdivide the image information of the total field of view and the image information of the partial fields of view into image blocks, wherein the calculating unit is configured to associate the first block with the second block while applying an association criterion so as to perform the combination only if the association criterion is met, and to provide the combined image information of the total field of view within the first block without any combination if the association criterion is not met.
 8. The device as claimed in claim 1, wherein the calculating unit is configured to perform stitching of the image information of the partial fields of view on the basis of the image information of the total field of view.
 9. The device as claimed in claim 1, wherein the calculating unit is configured to generate a depth map for the image information of the total field of view while using a first disparity between an image of the total field of view and an image of the first partial field of view and a second disparity between the image of the total field of view and an image of the second partial field of view.
 10. The device as claimed in claim 1, wherein the partial fields of view overlap within an overlap area within the total field of view, the calculating unit being configured to generate a depth map for the image information of the total field of view within the overlap area while using the image information of the first and second partial fields of view.
 11. The device as claimed in claim 1, comprising an optical image stabilizer for image stabilization along a first image axis by generating a first relative movement between the image sensor and the array and for image stabilization along a second image axis by generating a second relative movement between the image sensor and the array.
 12. The device as claimed in claim 1, further comprising focusing device comprising at least one actuator for adjusting a focus of the device, said actuator being configured to provide a relative movement between at least one optic of one of the optical channels and the image sensor.
 13. A supplementation device comprising a device as claimed in claim 1 and configured to be coupled to a camera so as to acquire therefrom the image information of the total field of view.
 14. A multi-aperture imaging device comprising: an image sensor; and an array of optical channels, each optical channel comprising an optic for projecting at least one partial field of view of a total field of view onto an image sensor area of the image sensor; a first optical channel of the array being configured to image a first partial field of view of the total field of view, a second optical channel of the array being configured to image a second partial field of view of the total field of view, and a third optical channel being configured to fully image the total field of view; wherein the device comprises a calculating unit configured to acquire image information of the first and second partial fields of view on the basis of the imaged partial fields of view, and to acquire image information of the total field of view on the basis of the imaged total field of view, and to combine the image information of the partial fields of view with the image information of the total field of view so as to generate combined image information of the total field of view; and the calculating unit is configured to subdivide the image information of the total field of view and the image information of the partial fields of view into image blocks and to associate, block by block, image information, which is comprised by a first image block of the total field of view, with matching image information of a second image block of the first or second partial fields of view so as to increase, by combining the first and second image blocks, a degree of scanning of the image information of the total field of view in the combined image information; or wherein the device is configured to acquire image information of the total field of view comprising a first degree of scanning from the sensor, to acquire the image information of the first or second partial field of view comprising a second degree of scanning, which is larger than the first degree of scanning, from the sensor, and to provide the combined image information of the total field of view comprising a third degree of scanning larger than the first degree of scanning.
 15. The multi-aperture imaging device as claimed in claim 14, comprising beam-deflecting device for jointly deflecting an optical path of the first and second optical channels.
 16. The multi-aperture imaging device as claimed in claim 14, wherein an arrangement of optics for capturing the first and second partial fields of view in the array is symmetric in relation to a location of the optic for imaging the total field of view; or wherein an arrangement of image sensor areas for imaging the first and second partial fields of view is symmetric in relation to a location of the image sensor area for imaging the total field of view.
 17. The multi-aperture imaging device as claimed in claim 14, wherein an image format of the total field of view corresponds to a redundancy-free combination of the imaged first partial field of view and the imaged second partial field of view.
 18. The multi-aperture imaging device as claimed in claim 14, comprising the calculating unit, and wherein the calculating unit is configured to associate the first block with the second block while applying an association criterion so as to perform the combination only if the association criterion is met, and to provide the combined image information of the total field of view within the first block without any combination if the association criterion is not met.
 19. The multi-aperture imaging device as claimed in claim 14, comprising the calculating unit, and wherein the calculating unit is configured to perform stitching of the image information of the partial fields of view on the basis of the image information of the total field of view.
 20. The multi-aperture imaging device as claimed in claim 14, comprising the calculating unit, and wherein the calculating unit is configured to generate a depth map for the image information of the total field of view while using a first disparity between an image of the total field of view and an image of the first partial field of view and a second disparity between the image of the total field of view and an image of the second partial field of view.
 21. The multi-aperture imaging device as claimed in claim 14, comprising the calculating unit, and wherein the partial fields of view overlap within an overlap area within the total field of view, the calculating unit being configured to generate a depth map for the image information of the total field of view within the overlap area while using the image information of the first and second partial fields of view.
 22. The multi-aperture imaging device as claimed in claim 14, configured to acquire the image information of the total field of view comprising the first degree of scanning from the image sensor, wherein the second degree of scanning is at least 30% larger, along first and second image directions, than the first degree of scanning.
 23. The multi-aperture imaging device as claimed in claim 14, configured to acquire image information of the total field of view, which is captured by the third optical channel, from the image sensor at a first image resolution, the multi-aperture imaging device comprising display device and being configured to display the image information at the first image resolution at the most.
 24. The multi-aperture imaging device as claimed in claim 23, configured to output a video signal of the total field of view on the display device on the basis of consecutive images of the total field of view.
 25. The multi-aperture imaging device as claimed in claim 14, comprising beam-deflecting device for shared deflecting of an optical path of the optical channels, and comprising an optical image stabilizer for image stabilization along a first image axis by generating a first relative movement between the image sensor, the array and the beam-deflecting device, and for image stabilization along a second image axis by generating a second relative movement between the image sensor, the array and the beam-deflecting device.
 26. The multi-aperture imaging device as claimed in claim 25, wherein the optical image stabilizer is configured to provide a first relative movement for the first and second optical channels and a second relative movement for the third optical channel.
 27. The multi-aperture imaging device as claimed in claim 25, wherein the first relative movement comprises at least one of a translational relative movement between the image sensor and the array, a translational relative movement between the image sensor and the beam-deflecting device, and a translational relative movement between the array and the beam-deflecting device, and wherein the second relative movement comprises at least one of a rotational movement of the beam-deflecting device, a translational relative movement between the image sensor and the array, and a translational relative movement between the array and the beam-deflecting device.
 28. The multi-aperture imaging device as claimed claim 25, further comprising an electronic image stabilizer for image stabilization of the first optical channel of the array along the first and second image axes.
 29. The multi-aperture imaging device as claimed in claim 28, wherein the electronic image stabilizer is configured to stabilize the first optical channel to a first extent along the first and second image axes, and further is configured for image stabilization of the second optical channel to a second extent along the first and second image axes.
 30. The multi-aperture imaging device as claimed in claim 28, wherein the optical image stabilizer is configured to perform optical image stabilization such that said optical image stabilization is related to an image of a first one of the partial fields of view, wherein the electronic image stabilizer is configured to stabilize an image of a second partial field of view in relation to the image of the first partial field of view.
 31. The multi-aperture imaging device as claimed in claim 28, wherein the optical image stabilizer is configured to stabilize an image of the imaged partial field of view of a reference channel from a group comprising the first optical channel and the second optical channel, and wherein the electronic image stabilizer is configured to perform image stabilization in a channel-specific manner for optical channels which differ from the reference channel, the multi-aperture imaging device being configured to stabilize the reference channel in an exclusively optical manner.
 32. The multi-aperture imaging device as claimed in claim 28, wherein the electronic image stabilizer is configured to perform image stabilization for each optical channel in a channel-specific manner.
 33. The multi-aperture imaging device as claimed in claim 32, wherein the electronic image stabilizer is configured to perform channel-specific electronic image stabilization in each channel in accordance with a specified functional correlation which depends on the relative movements between the image sensor, the array and the beam-deflecting device.
 34. The multi-aperture imaging device as claimed in claim 33, wherein the functional correlation is a linear function.
 35. The multi-aperture imaging device as claimed in claim 28, wherein the optical image stabilizer is configured to provide the optical image stabilization along one of the image directions on the basis of a rotational movement of the beam-deflecting device, the functional correlation being an angular function which projects an angle of rotation of the beam-deflecting device onto an extent of the electronic image stabilization along the image direction.
 36. The multi-aperture imaging device as claimed in claim 28, wherein the electronic image stabilizer is configured to identify a matching feature in a first partial image of a first partial field of view at a first point in time and at a second point in time, and to provide the electronic image stabilization on the basis of a comparison of movements of the feature in the first image.
 37. The multi-aperture imaging device as claimed in claim 28, further comprising focusing device comprising at least one actuator for adjusting a focus of the device, said actuator being configured to provide a relative movement between at least one optic of one of the optical channels and the image sensor.
 38. The multi-aperture imaging device as claimed in claim 35, wherein the focusing device is configured to provide a third relative movement for the first and second optical channels and a fourth relative movement for the third optical channel.
 39. The multi-aperture imaging device as claimed in claim 14, wherein the image sensor areas are arranged on the image sensor along a line extension direction, and wherein the image sensor areas exhibit, along an image direction perpendicular to the line extension direction, dimensions which are identical within a tolerance range of 20%.
 40. The multi-aperture imaging device as claimed in claim 14, wherein the first and second optical channels are part of a group of optical channels configured to image a partial field of view of the total field of view, respectively, said group of optical channels being configured to jointly fully image the total field of view.
 41. The multi-aperture imaging device as claimed in claim 39, wherein the group of optical channels is configured to capture precisely two partial fields of view.
 42. The multi-aperture imaging device as claimed in claim 14, wherein a first partial image and a second partial image, which represent the image information, provided by the image sensor, of the first and second partial fields of view, comprise a same dimension, along a first image direction, as a total image which represents image information, provided by the image sensor, of the total field of view, and exhibit, along a second image direction, a different dimension as compared to the total image.
 43. The multi-aperture imaging device as claimed in claim 14, configured to provide the imaged first and second partial fields of view comprising single-color luminance information.
 44. The multi-aperture imaging device as claimed in claim 14, configured as a mobile phone, a smartphone, a tablet, or a monitor.
 45. A multi-aperture imaging device comprising: an image sensor; and an array of optical channels, each optical channel comprising an optic for projecting at least one partial field of view of a total field of view onto an image sensor area of the image sensor; a first optical channel of the array being configured to image a first partial field of view of the total field of view, a second optical channel of the array being configured to image a second partial field of view of the total field of view, and a third optical channel being configured to fully image the total field of view; wherein an image format of the total field of view corresponds to a redundancy-free combination of the imaged first partial field of view and the imaged second partial field of view.
 46. A method of providing a device, comprising: providing an image sensor; arranging an array of optical channels, so that each optical channel comprises an optic for projecting at least one partial field of view of a total field of view onto an image sensor area of the image sensor, so that a first optical channel of the array is configured to image a first partial field of view of the total field of view, and so that a second optical channel of the array is configured to image a second partial field of view of the total field of view; and arranging a calculating unit such that same is configured to acquire image information of the first and second partial fields of view on the basis of the imaged partial fields of view and to acquire image information of the total field of view and to combine the image information of the partial fields of view with the image information of the total field of view so as to generate combined image information of the total field of view; and such that the device is configured to acquire the image information of the total field of view comprising a first degree of scanning, to acquire the image information of the first or second partial field of view comprising a second degree of scanning, which is larger than the first degree of scanning, and to provide the combined image information of the total field of view comprising a third degree of scanning larger than the first degree of scanning; or such that the calculating unit is configured to subdivide the image information of the total field of view and the image information of the partial fields of view into image blocks and to associate, block by block, image information, which is comprised by a first image block of the total field of view, with matching image information of a second image block of the first or second partial fields of view so as to increase, by combining the first and second image blocks, a degree of scanning of the image information of the total field of view in the combined image information.
 47. A method of providing a multi-aperture imaging device, comprising: providing an image sensor; and arranging an array of optical channels, so that each optical channel comprises an optic for projecting at least one partial field of view of a total field of view onto an image sensor area of the image sensor, and so that a first optical channel of the array is configured to image a first partial field of view of the total field of view, so that a second optical channel of the array is configured to image a second partial field of view of the total field of view, and so that a third optical channel is configured to fully image the total field of view; such that the device comprises a calculating unit configured to acquire image information of the first and second partial fields of view on the basis of the imaged partial fields of view, and to acquire image information of the total field of view on the basis of the imaged total field of view, and to combine the image information of the partial fields of view with the image information of the total field of view so as to generate combined image information of the total field of view; and the calculating unit is configured to subdivide the image information of the total field of view and the image information of the partial fields of view into image blocks and to associate, block by block, image information, which is comprised by a first image block of the total field of view, with matching image information of a second image block of the first or second partial fields of view so as to increase, by combining the first and second image blocks, a degree of scanning of the image information of the total field of view in the combined image information; or such that the device is configured to acquire image information of the total field of view comprising a first degree of scanning from the sensor, to acquire the image information of the first or second partial field of view comprising a second degree of scanning, which is larger than the first degree of scanning, from the sensor, and to provide the combined image information of the total field of view comprising a third degree of scanning larger than the first degree of scanning.
 48. A method of providing a multi-aperture imaging device, comprising: providing an image sensor; and arranging an array of optical channels, so that each optical channel comprises an optic for projecting at least one partial field of view of a total field of view onto an image sensor area of the image sensor, and so that a first optical channel of the array is configured to image a first partial field of view of the total field of view, so that a second optical channel of the array is configured to image a second partial field of view of the total field of view, and so that a third optical channel is configured to fully image the total field of view; such that an image format of the total field of view corresponds to a redundancy-free combination of the imaged first partial field of view and the imaged second partial field of view. 